Extend Cache runtime framework Phase 2: support curvine poc#5762
Merged
cheyang merged 11 commits intofluid-cloudnative:masterfrom Apr 16, 2026
Merged
Extend Cache runtime framework Phase 2: support curvine poc#5762cheyang merged 11 commits intofluid-cloudnative:masterfrom
cheyang merged 11 commits intofluid-cloudnative:masterfrom
Conversation
|
Skipping CI for Draft Pull Request. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #5762 +/- ##
==========================================
- Coverage 60.47% 58.46% -2.01%
==========================================
Files 465 473 +8
Lines 31148 32222 +1074
==========================================
+ Hits 18837 18839 +2
- Misses 10764 11836 +1072
Partials 1547 1547 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: xliuqq <xlzq1992@gmail.com> support app pod cv mount succeed fix mount todo: use bin/cv mount to filter already mount path master worker pod starts succeed set envs service name should be exposed Signed-off-by: liuzhiqiang <923463801@qq.com>
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Signed-off-by: xliuqq <xlzq1992@gmail.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| spec: | ||
| template: | ||
| spec: | ||
| containers: |
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Signed-off-by: xliuqq <xlzq1992@gmail.com>
xliuqq
changed the title
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
This PR extends the Cache runtime framework (Phase 2) to support a Curvine PoC, adding new component reconciliation logic (master/worker/client), runtime config generation/mounting, and accompanying GHA e2e manifests/scripts.
Changes:
- Add Curvine GHA e2e test assets (MinIO, Dataset/CacheRuntime/Class, mount script, read/write jobs).
- Implement CacheRuntime component reconciliation/status/volume/configmap sync scaffolding for Curvine-style topologies.
- Extend CacheRuntimeClass API/CRDs with
executionEntries(e.g.,mountUFS) and generate corresponding OpenAPI.
Reviewed changes
Copilot reviewed 42 out of 42 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| test/gha-e2e/curvine/write_job.yaml | Adds a write Job to validate writes via the mounted PVC. |
| test/gha-e2e/curvine/read_job.yaml | Adds a read Job to validate data persistence/visibility. |
| test/gha-e2e/curvine/test.sh | Adds an e2e orchestration script for Curvine runtime bring-up and verification. |
| test/gha-e2e/curvine/mount.yaml | Adds a ConfigMap with a mount script used by Curvine master to mount UFS. |
| test/gha-e2e/curvine/minio.yaml | Adds a MinIO Deployment/Service to back the S3 mount used by Curvine. |
| test/gha-e2e/curvine/minio_create_bucket.yaml | Adds a Job to create the required MinIO bucket. |
| test/gha-e2e/curvine/dataset.yaml | Adds a Dataset manifest configured for an S3 mount to MinIO. |
| test/gha-e2e/curvine/cacheruntime.yaml | Adds a CacheRuntime manifest that references the Curvine CacheRuntimeClass. |
| test/gha-e2e/curvine/cacheruntimeclass.yaml | Adds a CacheRuntimeClass defining component topology/templates and execution entries for Curvine. |
| pkg/utils/runtimes.go | Removes GetCacheRuntimeClass helper (migrated to engine method). |
| pkg/utils/kubeclient/configmap.go | Adds ConfigMap creation helper supporting OwnerReferences. |
| pkg/ddc/efc/runtime_info.go | Adds TODO while fetching owner dataset UID from runtime metadata. |
| pkg/ddc/cache/engine/worker.go | Implements worker setup reconciliation and status update logic. |
| pkg/ddc/cache/engine/volume.go | Implements PV/PVC create/delete for cache runtime fuse mount. |
| pkg/ddc/cache/engine/util.go | Adds helpers for naming, runtime config paths, and fuse mount paths. |
| pkg/ddc/cache/engine/ufs.go | Implements UFS mount execution via execution entries in master pod. |
| pkg/ddc/cache/engine/transform_worker.go | Adds worker component transformer from runtime/class to component value. |
| pkg/ddc/cache/engine/transform_master.go | Adds master component transformer from runtime/class to component value. |
| pkg/ddc/cache/engine/transform_client.go | Adds client transformer and fuse HostPath mount injections. |
| pkg/ddc/cache/engine/transform.go | Implements core transform pipeline and common volume/env injection. |
| pkg/ddc/cache/engine/sync.go | Implements runtime config ConfigMap sync during reconcile. |
| pkg/ddc/cache/engine/status.go | Implements master/worker/client status calculations via component helpers. |
| pkg/ddc/cache/engine/shutdown.go | Implements shutdown fuse cleanup via ctrl helper. |
| pkg/ddc/cache/engine/setup.go | Wires runtimeClass retrieval and configmap creation + UFS mount into setup. |
| pkg/ddc/cache/engine/runtime.go | Adds engine-level runtimeClass and runtimeInfo retrieval helpers. |
| pkg/ddc/cache/engine/master.go | Implements master setup reconciliation/status update and master pod info helper. |
| pkg/ddc/cache/engine/fileutils.go | Adds exec/mount helper for running commands in pods with timeout and redaction. |
| pkg/ddc/cache/engine/engine.go | Adds runtimeInfo field to CacheEngine. |
| pkg/ddc/cache/engine/dataset.go | Adds dataset mount option/encrypt option materialization from Secrets. |
| pkg/ddc/cache/engine/cm.go | Implements runtime config ConfigMap generation + class extra configmaps creation. |
| pkg/ddc/cache/engine/client.go | Implements client setup reconciliation and status update logic. |
| pkg/ddc/cache/component/statefulset_manager.go | Adds StatefulSet reconciliation and component status construction. |
| pkg/ddc/cache/component/daemonset_manager.go | Adds DaemonSet reconciliation and component status construction. |
| pkg/ddc/cache/component/component_manager.go | Adds workload-type dispatcher and common label helpers for components. |
| pkg/ddc/base/runtime.go | Updates CacheRuntime handling in runtime info building and adds TODOs. |
| pkg/common/label.go | Adds cache runtime label/annotation constants. |
| pkg/common/constants.go | Adds DefaultNameSpace constant. |
| pkg/common/cacheruntime.go | Adds cache runtime component/config structs for generated runtime config. |
| config/crd/bases/data.fluid.io_cacheruntimeclasses.yaml | Extends CRD schema with executionEntries. |
| charts/fluid/fluid/crds/data.fluid.io_cacheruntimeclasses.yaml | Mirrors CRD schema changes for Helm chart packaging. |
| api/v1alpha1/openapi_generated.go | Adds OpenAPI schemas for ExecutionEntries / ExecutionCommonEntry. |
| api/v1alpha1/cacheruntimeclass_types.go | Adds ExecutionEntries types to API. |
Comments suppressed due to low confidence (5)
test/gha-e2e/curvine/mount.yaml:1
- The second validation checks
$mountPointagain but the message says it's validatingpath. This makes the script fail to detect an emptypath. Change the second condition to check$path.
test/gha-e2e/curvine/test.sh:1 .status.succeeded/.status.failedcan be empty before the controller sets them. Using-ne/-eqon an empty string will causebash: integer expression expectedand potentially break the test loop. Normalize empties to 0 (e.g., parameter expansion defaults) before numeric comparisons.
test/gha-e2e/curvine/minio.yaml:1- Using
hostPort: 9000can cause scheduling failures or port conflicts in multi-node/shared CI clusters (and is generally discouraged for in-cluster services). Prefer ClusterIP-only access (remove hostPort) or constrain scheduling explicitly if hostPort is truly required.
test/gha-e2e/curvine/write_job.yaml:1 mkdir /data/foowill fail if the directory already exists (e.g., reruns or partial cleanup), which can make the e2e test flaky. Use an idempotent directory creation (e.g.,mkdir -p).
test/gha-e2e/curvine/minio_create_bucket.yaml:1mc mb myminio/testfails if the bucket already exists, which can make retries/reruns flaky. Consider using the MinIO client option to ignore existing buckets (or check existence before creation).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
cheyang
reviewed
Apr 14, 2026
cheyang
reviewed
Apr 14, 2026
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Signed-off-by: xliuqq <xlzq1992@gmail.com>
Signed-off-by: xliuqq <xlzq1992@gmail.com>
|
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: cheyang The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ⅰ. Describe what this PR does
cache runtime support curvine poc.
Ⅱ. Does this pull request fix one issue?
part of #5412
Ⅲ. List the added test cases (unit test/integration test) if any, please explain if no tests are needed.
e2e test
Ⅳ. Describe how to verify it
Ⅴ. Special notes for reviews
minimal implementation for curvine poc, left runtime/runtimeClass/dataset some fields unprocessed.